Estimating Likelihoods for Topic Models

Topic models are a discrete analogue to principle component analysis and independent component analysis that model topic at the word level within a document. They have many variants such as NMF, PLSI and LDA, and are used in many fields such as genetics, text and the web, image analysis and recommender systems. However, only recently have reasonable methods for estimating the likelihood of unseen documents, for instance to perform testing or model comparison, become available. This paper explores a number of recent methods, and improves their theory, performance, and testing. ∗. Prepared for The 1st Asian Conference on Machine Learning, 2009, Nanjing China. The original publication is available at www.springerlink.com. †. Also a fellow at Helsinki Institute of IT.